Shape Detection by Packing Contours

SHAPE DETECTION BY PACKING CONTOURS Qihui Zhu Jianbo Shi Humans have an amazing ability to localize and recognize object shapes from natural images with various complexities, such as low contrast, overwhelming background clutter, large shape deformation and signicant occlusion. We typically recognize object shape as a whole the entire geometric conguration of image tokens and the context they are in. Detecting shape as a global pattern involves two key issues: model representation and bottom-up grouping. A proper model captures long range geometric constraints among image tokens. Contours or regions that are grouped from bottom-up capture correlations of individual image tokens, and often appear as half complete shapes that are easily recognizable. The main challenge of incorporating bottom-up grouping arises from the representation gap between image and model. Fragmented image structures usually do not correspond to semantically meaningful model parts. This thesis presents Contour Packing, a novel framework that detects shapes in a global and integral way, effectively bridging this representation gap. We rst develop a grouping mechanism that organizes individual edges into long contours, by encoding Gestalt factors of proximity, continuity, collinearity, and closure in a graph. The contours are characterized by their topologically ordered 1D structures, against otherwise chaotic 2D image clutter. Used as integral shape matching units, they are powerful for preventing accidental alignment to isolated edges, dramatically reducing false shape detections in clutter. We then propose a set-to-set shape matching paradigm that measures and compares holistic shape congurations. Representing both the model and the image as a set of contours, we seek packing a subset of image contours into a complete shape formed by model contours. The holistic conguration is captured by shape features with a large spatial extent, and the long-range contextual relationships among contours. The unique feature of this approach is the ability to overcome unpredictable contour fragmentations. Computationally, iv set-to-set matching is a hard combinatorial problem. We propose a linear programming (LP) formulation for efciently searching over exponentially many contour congurations. We also develop a primal-dual packing algorithm to quickly bound and prune solutions without actually running the LPs. Finally, we generalize set-to-set shape matching on more sophisticated structures aris ing from both the model and the image. On the model side, we enrich the representation by compactly encoding part conguration selection in a tree. This makes it applicable to holistic matching of articulated objects with wild poses. On the image side, we extend contour packing to regions, which has a fundamentally different topology. Bipartite graph packing is designed to cope with this change. A formulation by semidenite program ming (SDP) provides an efcient computational solution to this NP-hard problem, and the exibility of expressing various bottom-up grouping cues.